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ABSTRACT 

The evolution of California's school accountability 
program, its current status, and some future directions are 
discussed. Public recognition of the r-^ed for school improvement 
resulted in California's Hughes Hart bducation Reform Act (SB 813) of 
1983, leading to the developraent of the California accountability 
program. Program components include: (1) an annual statewide 
performance report' on the status of uniform educational quality 
indicators; (2) annual individual school progress reports; (3) 
suggested goals for improvement in performance; (4) school 
self-reports on local quality indicators; and (5) an active program 
of school recognition. School performance reports are considered 
public information and are used in screening schools for the 
California School Recognition Program to reward achievement. New 
directions for the accountability program include: th^* preparation 
and distribution of school accountability report cards each year; the 
development of strategies for identifying and improving at-risk 
schools; and the development of a state plan for program improvement 
to encompass accountability standards for school programs under the 
federal reauthorization of Chapter 1 (Public Law 100-297, 1988). 
California's schools are presently undergoing enormous changes. 
However, despite these changes, accountability has proven to be a 
continuing theme in California's educational policy discussions. 
Accountability, conceived as the measurement of performance and the 
setting of goals, is providing for the effective and fair use of 
school resources. (SLD) 
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Assessing Educational Performance: 
California's School Quality Indicator System 



School accountability continues to interest lawmakers, educators and the 
public. For example, the U, S, Department of Education (I988, p. iii) 
opines that "citizens want to know in concrete termij what they are 
receiving in return for their hard earned tax dollars invested in public 
education," The Council of Chief State School Officers (I987) documented ^5 
states with integrated accountability programs which collect and report 
performance information, either about student achievement, course taking, 
school resources, processes, or background data on students and 
communities, A survey of the National Conference of State Legislatures 
found that accountability was identified as a top priority by education 
committee chairmen in 3' states, (Mirga, I989) The reasons for this 
interest in accountability appear to be associated both with school finance 
and in a concern for disadvantaged students. With the growth in states' 
education budgets has come a greater demand for improved school 
performance. At the same time there is sensitivity that the needs of at 
risk students are not being sufficiently met by the schools, 

California first implemented an integrated accountability program in 
1983 with uniform definitions of quality indicators^ annual publication of 
school performance reports, timelines for meeting improvement targets, and 
programs to recognize schools, (Fetler, I986) The objectives of this paper 
are to describe the evolution of Cal i f ornia ' s^cCountaSTl i ty program, to 
summarize its current status, and to discuss some possible future 
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directions, including initiatives for working with disadvantaged students 
and low performing schools* 

Evolution and Current Status, During the early 1980s public 
recognition of the need for school improvement resulted in California's 
Hughes Hart Education Reform Act (SB 8l3) of 1983* This legislation 
encouraged higher graduation standards and better student discipline, and 
provided incentives for student, teacher and school performance, and for 
increased instructional time. At the same time that these costly incentive 
programs were being implemented, there were also a number of fiscal 
concerns, A I987 study by Policy Analysis for California Education (PACE) 
described California's school finance at this time as "unstable and 
uncertain." Tax reform had severely limited the ability of school 
districts to generate revenues locally, and a 1979 constitutional 
expenditure limitation initiative threatened to hold down spending in the 
years ahead, A program to hold schools accountable would address the 
perceived need to raise academic standards and would provide additional 
justification for school funding. The stated purposes of California's 
accountability program are "to allow educators to determine the success of 
their own school programs, sustain support for the reform movement by 
demonstrating such success, recognize schools for their progress and 
achievements, and discover how to use the resources available for education 
in the most effective manner possible," (California State Department of 
Education, I988) 



The components of the accountability program include an annual 

statewide performance report on the status of uniform educational quality 
^ A 
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indicators, annual individual school reports of progress, suggested goals 
for improvement, school self reports on local quality indicators, and an 
active program of school recognition. The indicators are intended to be 
important "bottom line" measures of performance, to be used by school 
officials in setting priorities and making decisions. Timelines are set 
for the attainment of state and local goals for each indicator. Judgement 
about school performance are made on the basis of comparisons of the school 
with itself over time (trends), of the school with all other schools 
statewide, and of the school with other demograph i ca 1 1 y similar schools. 
The performance report and its accompaniments were implemented at the 
initiative of the Superintendent of Public Instruction, Bill Honig, and 
were not rooted in legislation. 

The school performance report provides for comparisons of the school 
with itself over time (trends), cf the school with all other schools 
statewide (undifferentiated norms), and of the school with demographical ly 
similar schools (differentiated norms). The principle underlying 
differentiated nc ns is the subdivision of the population of schools to 
allow comparisons of schools with reference groups of similar 
characteristics. This process requires the construction of a composite SES 
index from information describing the school's demographics, including 
percent of families receiving AFDC, parent education or occupation, percent 
of limited English speaking students, and student mobility. Schools are 
ranked on the basis of the composite SES index. The comparison group for 
any particular school consists of the ten percent of schools with 
immediately higher or lower SES indexes. Each school is at the median of 
its group in terms of the SES index, but its academic performance may be 
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relatively nigh or low. After identifying the comparison group, a 
percentile rank is computed by the usual procedure, A more detailed 
description of comparison group ranks can be found in Fetler (1989), 

Currently every public school receives an annual performance report. 
This is a booklet containing a discussion of the accountability program, an 
explanation of the performance indicators, statewide results and goals, a 
display of the indicators for that particular school, and a discussion of 
the concept of locally produced reports. Reports for high schools contain 
information on academic course enrollments, California Assessment Program 
(CAP) achievement scores, attendance, dropouts, the Scholastic Aptitude 
Test, the American College Testing program, and enrollments at public 
postsecondary institutions, Much of the information is presented by sex, 
ethnic category, and overall. Reports for elementary and intermediate 
schools contain CAP achievement and attendance information. The school 
performance reports are considered to be public information. Two copies of 
each school's report are sent to the district office. One of these copies 
is passed on to the school. County offices receive copies for all schools 
under their jurisdiction. The local districts have about two weeks to 
examine the reports before they are released to the media. 

One use of the school performance reports is in the screening of 
schools for the California School Recognition Program (I985) • The 
recognition accorded under this program is ceremonial and is intended to 
reward achievement, to motivate other schools to strive for excellence, to 
increase local awareness of school efforts, and to provide models of 
successful practices. Distinguished schools are selected annually and 
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represent the best all around schools in the state. An initial 
quantitative screening identifies schools that perform wel 1 relative to 
other comparable schools, or show unusual improvement on the various 
quality indicators- Those who survive the screening are invited to fill 
out applications which address curriculum, instructional practices, 
improvement efforts, school culture and student outcomes. These 
applications are scored by trained reviewers and site visits for the best 
applicants are conducted, A flr.wi comprehensive review of all available 
information precedes the selection of the winners. The selection is marked 
during an annual awards ceremony. The ceremony for high schools and middle 
schools is scheduled for even numbered years and for elementary schools on 
odd numbered years. Roughly ten percent of schools are recognized each 
year , 

New Directions, Public concern about school performance and 
improvement resulted in three accountability efforts mandated in I988, 
Proposition 98, the school funding initiative constitutional amendment, 
which provided for stable growth in funding, also requires school districts 
to prepare and distribute school accountability report cards each year. 
Assembly Bill 9 (Chapter 832, Statutes of I988) required the development of 
criteria for identifying and strategies for improving at risk schools. The 
federal reauthorization of Chapter 1 (Public Law 100-297i '988) required 
development of a state plan for program improvement which encompassed 
accountability standards for school programs. 

Proposition 98 established a task force on instructional improvement 
to make recommendations for a model school accountability report card. 
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With a majority of practicing classroom teachers, the task force also 
included school administrators, board members, classified employees, 
research specialists, and parents. The Superintendent of Public 
Instruction appointed and consulted with the task force to develop model 
report card, which was presented to the State Board of Education for 
adoption. 

The language of Proposition 98 requires that the report card include 
assessments of: student achievement, reduction in dropout rates, estirated 
expenditures per student and types of services funded, reduction of class 
size and teaching loads, assignment of teachers outside their areas of 
competence, quality of textbooks, availability of qualified counselors, 
availability of qualified substitute teachers, school safety and 
cleanliness, adequacy of teacher evaluation and opportunities for 
professional improvement, classroom discipline and climate, teacher and 
staff training and curriculum improvement, and quality of school 
instruction and leadership. The initiative states that the governing board 
of each school district must implement by September 30, 1989 for each 
school an accountability report card, triennial ly compare its 
accountability report card with the State's model, annually issue a report 
card for each school in the district, publicize such reports, and notify 
parents that copies are available on request. 



Assembly Bill 9 established an advisory task force on at-risk schools 
in order to develop criteria for identifying at risk schools and 
strategies for improving them. The task force was comprised of 13 members: 
seven appointed by the Governor, two by the Speaker of the Assembly, two by 
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the Senate Rules committee, and two appointed by the v .-rintendent of 
Publ ic I nstruction. 

The task force report (California, 1989a) discussed a number of issues 
relatins^ to the identification of at-risk schools. In order to provide for 
accurate and fair identification it was considered desirable to use as much 
high quality information as possible. However, the amount of information 
to be used is limited by various practical considerations including the 
local burden of providing data and the quality of existing data. After 
examing a large list of potential identification indicators the task force 
recommended for secondary schools using achievement results from the 
California Assessment Program, rate of actual attendance, rate of 
completion of the University of California academic course sequence 
required for admission, and the dropout rate for grades 10 - 12, The 
recommendation for elementary schools included achievement results and the 
rate of actual attendance. Other indicators could be used when they are 
available and of sufficient quality, including incidents of violence at the 
school, reclassification of limited English proficient students, and 
dropout rates for grades 7 " 9' 

The task force recommended that the identification process should 
begin with an evaluation of the status of each of the indicators. An 
indicator is considered to be at-risk if it falls below a threshold which 
defines the bottom five percent of all schools two years prior to enactment 
of legislation. If the indicator is below this threshold for at least two 
of the last three years, it is considered to be at-risk. If the indicator 
is in the bottaTi 25 percent of schools ranked statewide and in the bottom 
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25 percent relative to demographical ly similar schools, and it does not 
meet specific growth targets used in California's accountability program, 
it also is considered to be at-risk. 

The at-risk status of a school depends on the results for the various 
indicators. A school is identified if it was at-risk on CAP, regardless of 
the other criteria. Additionally, those elementary schools which are 
at-risk on attendance and were in the bottom ten percent statewide on CAP 
are identified. Those secondary school s' wh i ch are at-risk on at least two 
of the three other indicators (dropout, attendance, academic courses) and 
were in the bottom ten percent on CAP statewide are also identified. 

The task force recommended an intervention process which progresses 
through three stages of increasing assistance, monitoring, and control. 
The first stage consists of a two year period in which the identified 
school develops and implements its own plan for improvement with help from 
the district, the state and various independent organizations. The second 
phase begins when the school has not improved its performance over a two 
year period. Activities in the second phase include the appointment of an 
external support team, development of recommendations for changes in the 
schools improvement plan, the availabilit> of matching grants for training 
and other improvements, and tne continuation of assistance from the first 
stage. The lack of improvement for two further years triggers the last 
stage which involves the appointment by the state Superintendent of a 
trustee with broad legislatively defined powers to transfer personnel, 
cancel contracts, to revise budgets, and to stay or rescind actions of the 
local board. 
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The reauthorization of Chapter 1 called for greater accountability for 
Che improvement of educational opportunities of educationally deprived 
children in order to assure their success in regular school programs, the 
attainment of grade level proficiency, and improved achievement in basic 
and more advanced skills. The mechanism for implementing accountability is 
the State Plan for Program Improvement which is developed with the 
assistance of a committee of practitioners. Criteria included in the plan 
are used to evaluate Chapter 1 and State Compensatory Education schools to 
determine which sites are not providing an adequate program. 

The state plan (California, 1989b) includes objective measures ^rt6 
standards to assess student performance, the process for the joint 
Department and local development of improvement plans to attain 
satisfactory student progress, a timetable for developing and implementing 
improvement plans, and program of assistance to be provided to identified 
schools. Performance measures which are available on a uniform and 
statewide basis include both California Assessment Program achievement 
results in grades 3> 6 and 8 and locally administered norm referenced test 
results from the Chapter 1 evaluation system. Because the California 
Assessment Program tests are built around the state's curriculum 
frameworks, these test results should provide information on success in the 
regular program. Districts annually identify schools with insufficient 
improvement or declining achievement, using the criteria set forth in the 
State Plan, and provide the resources to facilitate improvement. If the 
identified schools do not improve according to the State Plan's timeline, 
the district works with the school to develop a plan of action. If the 
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planned program proves ineffective, the Department and the district work 
together until the expected academic progress is attained. 



A summary report of CAP and norm referenced ^-^^st results was compiled 
by the Department and sent to local districts in order to implement the 
identification process. The summary report is considered to be non-bind>ng 
advice to local districts for identifying school programs. The state 
defines the statewide targets for improvement of three scaled score points 
per year on CAP, established in the School Performance Reports for each 
grade and subject area, as the standard for making substantial progress 
toward meeting desired outcomes. CAP data are provided f )r the subgroup of 
students receiving Chapter 1 services for two consecutive years. If the 
average performance across grades and subjects of this subgroupis less than 
three scaled score points the school is identified as not meeting the 
standard established in the state plan. The state plan also defines a 
positive change in measured achievement from the pretest to the posttest as 
meeting the standard that students should achieve more than would be 
expected without the compensatory education program. Schools annually 
submit norm refer .need test (NRT) data for students in compensatory 
education programs. These test data are converted to normal curve 
equivalent scores (NCE) and the gain from the pre- to the posttest is 
computed. If the gain is is more than one standard error less than zero, 
the school is identified as not meeting the state* s standard for the NRT 
resul ts. 



Selected Issues for Identifying Low Performing Schools. Language in 
California's Proposition 98, Assembly Bi 1 1 9 and in the federal 
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reauthorization of Chapter 1 reflects a perceived need to quantify and 
judge school performance. At the same time, it is difficult to devise 
practical criteria to classify schools, (Oakes, I986; Murnane, 198?; U»S» 
Department of Education, I988) The information which is available for 
evaluating school performance is rarely perfect and its quality can be 
affected by the burdens of data collection on providers, incentives for 
biased reporting, and the use of less than exact models of the educational 
process. Even so, policy makers are sometimes compelled to make the best 
of the data that are available. Selected issues of criteria development 
which have arisen in California are discussed below, including, the need to 
go beyond achievement test results, the types of comparisons that can be 
made, the tension between state needs and local autonomy, the use of key 
versus supplemental indicators, the amount of information available for 
high schools versus elementary schools, and the difficulty of making 
judgement about small schools. 

Exclusive reliance on test scores can cause problems. Criticisms of 
existing tests include the inability to measure directly the full range of 
achievement, particularly higher order or performance skills, the 
disparities between the content of tests and the content in widely used 
textbooks, the possible narrowing of the curriculum to reflect merely the 
content of tests, and confusion over the interpretation of results. 
Information on aspects of performance in addition to achievement may help 
to identify low performing schools. Other possibly useful indicators could 
for example include student dropout rates and attendance, the numbers of 
students enrolled in certain courses and student attitudes and aspirations. 
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Taking account of student demographics would seem to be a fair way of 
comparing schools. Yet it is important to avoid the suggestion that low 
performance is acceptable for certain groups of students. The comparison 
group ranks used in the Performance Reports are a way of recognizing that 
some schools have greater obstacles to overcome in teaching disadvantaged 
students than others. Using the comparison group rank a school can judge 
its performance in relation to other schools with similar student 
populations. Unfortunately, it is possible for a school to compare well 
with others in its group, and yet have very low results. These ranks are 
not intended to excuse low performance and should be interpreted in the 
context of unadorned statewide comparisons and score trends. 

Local and state educators have different needs. At the state level 
there is need to see that allocated tax dollars are well spent in all 
schools, to encourage equality of opportunity for a good education 
statewide, and to institute remedies where inequalities exist. Comparable 
information provides a more equitable basis for judging the educational 
opportunities of students statewide. Existing inequalities can be located 
and remedies instituted. Information is more reliable when that 
information has been obtained under standardized and controlled conditions. 
The quality and credibility of the process for identifying at risk schools 
is reduced to the extent that differences across schools or districts in 
definitions or procedures for obtaining information influence the 
measurement of performance. On the other hand, information needed locally 
may be too unique for statewide comparability. Local interest in and 
ownership of information is probably related to the quality of that 
information. Local educators may also be concerned that the public release 
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unproductive debate. 

Testing programs, for example, tend to reflect either state or local 
needs. Different kinds of tests measure the attainment of different 
objectives and can result in different judgments of performance. For 
example, minimum competency or proficiency tests often assess only the 
skills required for basic literacy. By contrast, commercially available 
standard tests go beyond basic literacy 'to reflect the content of 
textbooks commonly used nationwide. These commercially available tests are 
often used to provide student diagnostic information and for making 
decisions about services, e,g,, placement in compensatory education or 
gifted and taiented programs. More academically oriented tests, such as 
the Advanced Placement examinations of the College Board are oriented 
towards the content of first year college courses. Recently developed CAP 
tests are constructed around California's curriculum frameworks. The CAP 
tests are meant to encourage districts to implement the frameworks. 

It may be possible to measure the major aspects of school performance 
with a few key indicators, e,g,, achievement, enrollment in academic 
subjects, attendance, and dropout rates. Although other supplemental 
indicators could certainly be used, the/ may not lead to a different 
judgement of overall performance. More detailed information, e,g,, results 
for various student subgroups or finer breakouts of test scores, can 
provide more background for study and planning. Decisions about key versus 
supplemental indicators should take into account costs and benefits. 
Although it may be technically possible to collect and process large 
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amounts of information, the burden of reporting for schools and the need 
for timely and accurate processing will limit i rs practical usefulness. 

To the extent that the use of performance information affects those 
people who are measured, there can be unintended consequences. High stakes 
are involved in the public identification of high or low performance, and 
the selection of a school for intervention. Rewards or sanctions can 
affect behavior. The use of a commercially available standardized test to 
measure student achievement might result in the narrowing of curriculum to 
cover just the material in the test. Would increased funding for at risk 
schools provide practical means for improvement, or would it function more 
as an incentive to remain at risk? Do higher standards motivate students, 
or will they increase the risk of dropping out? Other possible problems, 
although in the past these have been highly infrequent, include 
inappropriate preparations for the test (e,g,, practicing on the test), 
irregularities in test administration (e,g, coaching, selective testing), 
and altering of student responses. 

When information about a school is based on the responses of a large 
pjmber of students, the influence of any particular individual on the 
result tends to be washed out. The resulting information can be 
interpreted in terms of the school and its general surroundings. By 
contrast, information about the performance of small schools is based on 
relatively few students. The characteristics of particular individuals can 
wei^more heavily at a small school • Therefore school performance across 
time tends to fluctuate more widely at a small school than at a larger 
school. One strategy to overcome this difficulty is to examine the 
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performance of small schools over a number of years using a moving average. 
Although a moving average is less susceptible to the influence of 
individual students, it does mean that more tJme is required to judge the 
progress. 

Conclusions 

California's schools are presently undergoing enormous changes. 
School enrollments are predicted to increase by at least 1^0,000 students 
per year. Ethnic students now comprise a Tiajority of the population and 
increasing numbers of students do not soaa*^. English or have limited English 
proficiency. Local agencies have lost effective control of funding, with 
more than two thirds of the money for education now coming from the state. 

Despite these changes accountability has proven to be a continuing 
theme in California's educational policy discussions. Proposition 98, the 
school funding initiative, provides schools with a stable source of 
funding, but requires that the public receive a report card on each 
school's performance. Concern with the inequality of educational 
opportunity at low performing schools produced Assembly Bill 3, which may 
stimulate additional legislation and action. The shift in emphasis of 
Chapter 1 programs, away from basic skills, towards a commitment to 
success in the regular program, and the identification of low performing 
programs, hints at a national impatience with Ihe attainments of 
compensatory education programs thus far. 
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One can speculate that educators, in working to provide all students 
with a quality edccstion, know that they are working for a good cause. Yet 
this should not divert attention from the need to direct and expend 
educational resources effectively and fairly. Accountability, conceived as 
the measurement of performance and the setting of goals, is one way to 
promote the effective and fair use of resources. 
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